Customers expect performance improvements for every new computer model. In the past, advances in solid state physics allowed increasing clock frequency from about 1 MHz around 1980 to several GHz today. However, currently, improvements of solid state physics have become more difficult. Increased speed of program execution may thus come from improved CPU structure, such as utilizing cache memory.
In general, cache memory can be hardware and/or software components of a computer that store data to server future data requests. The data stored in the cache memory can result, for example, from of an earlier computation, or be duplicated from data stored elsewhere. The data stored in the cache memory can be called cache contents, which can further be referred to as cache lines (i.e., a contiguous sequence of memory cells). A cache hit occurs when the requested data can be found in the cache memory. Cache hits are served by reading data from the cache memory, which can be served faster from the cache memory than re-computing a result or reading from a slower data store. A cache miss occurs when the requested data cannot be found in the cache memory.
An example of management of a cache memory will now be described. When execution of a new or first program begins by a computer, the cache memory is empty in the sense that none of the data items needed by the program (i.e., a working set) are available from the cache memory. In turn, when execution of the first program begins, a first access of a data item will cause a cache miss. To avoid subsequent cache misses, the cache memory is loaded with the data items needed by the program from a memory of the computer. Thus, as the first program executes, the data items needed by the program are loaded over time into the cache memory based on requests from that first program. This loading over time fails to the initial time consuming cache misses, which are not optimal for the first program.
Another example of management of a cache memory will now be described with respect to the computer and the first program described above. When an operating system of the computer performs a context switch (i.e., a switch from the first program to a second program), the cache memory will contain lines used by the first program (e.g., the working set of the first program) immediately after the context switch because the context switch itself does not change an actual cache content of the cache memory. That is, after the context switch, the cache memory does not contain a working set relevant to the second program; rather, the second program accesses data items in the memory and builds its working set while replacing older data (e.g., possibly the working set of the first program) in the cache memory. Thus, when execution of the second program begins, accesses by the second program to data items of the cache memory will therefore cause time consuming cache misses, which are not optimal for the second program.